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Key  Points 


Methodology  for  evaluating  bene  lit  of  increasing  assimilative  data  sources 


Abstract 

A  methodology  for  evaluating  the  science  benefit  of  adding  space  weather  sensor  data  from  a 
modest  number  of  small  satellites  to  the  Utah  Slate  University  Global  Assimilation  of 
Ionospheric  Measurements  -  Full  Physics  {GAIM-FP)  model  is  presented.  Three  orbital 
scenarios  are  presented,  two  focusing  on  improved  coverage  of  narrowly  specified  regions  of 
interest,  and  one  on  global  coverage  of  the  ionosphere  as  a  whole.  An  Observing  System 
Simulation  Experiment  (OSSE)  is  used  to  obtain  qualitative  and  quantitative  results  of  the  impact 
of  the  various  orbital  scenarios  on  the  ionospheric  specifications.  A  simulated  "truth'’  run  of  the 
ionosphere  is  obtained  from  a  first  principles  model  of  the  lonosphere/Plasmasphere  model 
(1PM)  and  used  to  generate  global  simulated  Global  Positioning  Satellite  Total  Electron  Content 
(GPS-TEC)  data  as  well  as  in-situ  plasma  density  observations.  Initially,  only  GPS  data  were 
assimilated  by  GAIM-FP  and  die  results  of  this  limited  run  were  compared  to  the  truth  run.  Next, 
the  simulated  in-sim  plasma  densities  corresponding  to  our  three  orbital  scenarios  were 
assimilated  together  with  the  GPS  data  and  the  results  were  compared  to  both  the  truth  run  and 
the  limited  GPS-TEC  only  GAIM-FP  run.  These  model  simulations  have  shown  that  adding  a 
constellation  of  small  satellites/sensors  in  addition  to  global  PEC  inputs  does  indeed  converge 


the  GAIM-FP  model  closer  to  “truth'’  in  the  situations  described. 
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41  L  Introduction 

42  in  recent  years,  there  have  been  constructed  or  proposed  space  sensor  networks  [Anderson  el  al, 

43  2002;  Barnhart  cl  al,  2007a,  2007b:  Barnhart,  2008;  Barnhart  et  al?  2009;  J  7 adimirova  ct  al, 

44  2011;  Dyrud  el  al  201 3]  designed  to  cover  selected  orbits  in  LEO  with  cither: 

45  i)  low-cost  redundant  "disposable”  spacecraft-as-scnsor  platforms  of  CubeSat  3U  size  or 

46  smaller,  or 

47  ii)  low -cost  low-SWAP  (Size,  Weight  and  Power )  sensors  designed  to  be  placed  on  as  many 

48  conventional  (ESPA-dass  or  larger)  satellites  as  possible, 

49  The  science  objective  of  many  of  these  missions  is  to  provide  a  dense  set  of  sensor  data 

50  parameters  lo  "fill  in  the  gaps”  of  the  relatively  sparse  coverage  afforded  by  conventional  multi- 

51  million-dollar  missions  \de  la  Beaujardiere,  2004]  which  produce  single-point  in-situ  or  remote 

52  measurements. 

53  Despite  the  lower  cost  of  small  satellites  and  low-SWAP  instrumentation,  limitations  include 

54  funding,  choice  of  orbital  parameters  in  launch  opportunities,  space  debris  mitigation,  and  data 

55  vo lume/bandwidth  consideration.  A  key  question  asked  by  funders  and  approvers  is  still  "how 

56  many  sate  Elites/sensors  are  enough?”  What  has  become  apparent  is  that  there  is  no  generally 

57  agreed  metric  for  determining  the  scientific  justification  side  of  this  argument,  and  we  propose 

58  one  methodology  to  obtain  quantitative  metrics,  which  may  assist  in  answering  that  question. 
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2.  Scientific  Problem  and  Background 


The  science  problem  that  we  identify  for  this  study  is  space  weather  forecasting,  particularly 
forecasting  of  plasma  irregularities  ("plasma  bubbles"')  that  cause  radio  and  GPS  scintillation. 
Such  scintillation  can  cause  loss  of  GPS  lock  (less  than  50%  availability  for  LPV-2QQ  during 
severe  scintillation  [ Seo ,  2010]),  loss  of  communications,  and  image  defocussing  in  synthetic 
aperture  radar.  (LPV  stands  for  Localizer  Performance  with  Vertical  Guidance.  It  refers  to  a 
precision  approach  system  for  aviation.  I  .PV-200  availability  indicates  the  probability  for  a  GPS- 
autopilot-equipped  plane  to  reach  a  200*  decision  height  (the  height  at  which  the  pilot  has  to 
make  the  I  and /go- around  decision  and  assume  manual  control).  It  is  one  metric  used  to  determine 
the  efficacy  of  GPS  locational  ability.)  Forecasting  (and  nowcasting)  ionospheric  conditions 
conducive  to  plasma  bubble  formation  would  therefore  seem  to  require  global  assimilative 
models  of  the  ionosphere  to  provide  baseline  conditions  in  the  regions  of  interest  and  it  is  on  this 
facet  that  wc  concentrate. 

Our  ability  to  specify  and  forecast  ionospheric  dynamics  and  ionospheric  weather  at  low  and  mid 
latitudes  is  strongly  limited  by  our  current  understanding  of  the  coupling  processes  in  the 
ionosphere- thermosphere  system  and  the  coupling  between  the  high  and  low  latitude  regions. 
Furthermore,  only  a  limited  number  of  observations  are  available  for  a  specification  of 
ionospheric  dynamics  and  ionospheric  weather  at  these  latitudes.  As  shown  by  meteorologists 
and  oceanographers,  the  best  specification  and  weather  models  are  physics-based  data 
assimilation  models  that  combine  the  observational  data  with  our  understanding  of  the  physics  of 
the  environment  [Daley*  1991).  Through  simulation  experiments  these  models  can  also  he  used 
to  study  die  sensitivity'  of  Ute  specification  accuracy  on  different  arrangements  of  observation 
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platforms  and  observation  geometries  and  can  provide  important  information  for  the  planning  of 
future  missions*  For  example,  these  studies  can  provide  information  about  the  number  of 
spacecraft  needed  to  improve  the  specification  or  evaluate  the  impact  of  different  observation 
geometries  on  the  accuracy  of  the  specification. 

3.  Instrument  and  Satellite  Concept 

Previous  efforts  In  flying  low-cost  space  weather  instruments  have  focused  on  low-impact 
secondary  payloads  riding  on  larger  (e*g*  ESPA-cIass)  satellites*  These  have  advantages,  in  that 
the  larger  satellites  tend  to  he  more  reliable  (through  extensive  heritage  and  testing)  and  have 
larger  link  budgets  and  power  margins.  However,  there  arc  higher  integration  costs  (particularly 
when  co-riding  with  high-value  primary  payloads),  longer  project  timescales,  and  more 
expensive  busses  (£I0M+).  They  also  provide  only  a  few  in-si tu  measurements  for  each  satellite, 
and  have  relatively  long  delays  before  resampling  the  same  dataspace. 

There  have*  however,  been  recent  advances  in  thinking  regarding  multiple  low-cost  redundant 
satellites  carrying  low-cost  sensors.  Such  a  large  constellation  has  many  applications:  treaty 
sentinels,  disaster  monitoring,  magnetospheric  observations,  solar  wind  measurements,  pollution 
monitoring  and  communications  research  to  name  but  a  few.  In  many  of  the  above  cases  we 
currently  undersample  the  data  field.  We  concentrate  our  approach  for  tills  paper  on  l he 
thermosphere/ionosphere  system,  ingesting  plasma  density  and  temperature  data  from  in-siiu 
measurements  into  an  assimilative  model,  although  the  general  methodology  is  applicable  to  any 
of  the  above  applications. 
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With  this  approach  there  are  a  number  of  suitable  instruments  with  low  SWAP  that  obtain  in-si tu 
ionospheric  parameters  (we  restrict  ourselves  to  in-si tu  measurements  for  this  study,  although  a 
similar  methodology  may  be  used  with  remote  measurements).  The  MESA  (Miniaturized 
Electrostatic  Analyzer)  instrument  [Enloe  ct  al,  2002]  is  a  bandpass  or  high -pass  energy  filter  (in 
either  of  two  configurations).  The  instrument  thus  measures  ion  or  electron  spectra  (convolved 
with  the  instrument  response  function)  from  which  plasma  density  and  temperature  can  be 
derived.  MESAs  have  tlown  on  MISSE-6,  MISSE-7  [Jenkins  et  al,  2009],  ANDE-2, 
FalconSAT-5.  STP-H4  and  STPSat-3,  and  are  rostered  to  fly  on  OTB  and  other  missions. 
Another  instrument  of  interest  is  WINCS  (Winds  Ions  Neutrals  Composition  Suite)  and  its  sister 
instrument  SWATS  (Small  Wind  And  Temperature  Spectrometer),  developed  by  die  Naval 
Research  Laboratory-  WINCS  and  SWATS  are  sensor  suites  measuring  ion  and  neutral  winds, 
temperature  and  composition  (http:// www.nrl.na vy.mil/ssd/branches/76  30/S  WATS).  More 
generic/traditional  Langmuir  probes  and  Retarding  Potential  Analyzers  are  also  instruments  that 
may  with  care  be  integrated  into  a  low  SWAP  package  (although  the  standoff  length  of  a 
Langmuir  probe  may  prove  to  be  challenging). 


4.  GAIM-FP  Comparison  Methodology 

Al  Utah  State  University,  we  have  developed  two  physics-based  Kalman-filier  data  assimilation 
models  for  the  Earth  ionosphere.  The  two  models  are  the  Gauss-Markov  Kalman  Filter  Model 
(GAIM-GM)  and  the  Full  Physics-Based  Kalman  Filter  Model  (GAIM-FP)  [Scherliess  et  aL, 
2006,  2009],  Both  models  are  pan  of  the  Global  Assimilation  of  Ionospheric  Measurements 
(GAIM)  project  [Schrnk  et  aL  2004],  Some  of  the  data  that  we  have  previously  assimilated  in 
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our  data  assimilation  models  include  in-si tu  electron  density  measurements  from  DMSP 
satellites,  boitomside  electron  density  profiles  from  ionosondes,  GPS-TEC  data  from  a  network 
of  up  to  1000  ground  stations,  ultraviolet  (UV)  radiances  from  the  SSUSI  (Special  Sensor 
Ultraviolet  Specirographic  Imager),  SSUL1  (Special  Sensor  Ultraviolet  Limb  Imager),  and 
LORA  AS  (Low  Resolution  Airglow  and  Aurora  Spectrograph)  instruments,  and  radio 
occultaiion  data  from  Cl  LAMP  (Challenging  Alinisaieilitc  Payload),  SAC-C  (Satellite  de 
Aplicaciones  Cientificas-C)  \Hajj  et  ah  2004],  JOX  (Ionospheric  Occultaiion  Experiment) 
[Straus  et  al.  2003],  and  the  COSMIC  (Constellation  Observing  System  for  Meteorology  , 
Ionosphere  and  Climate)  |  Roc  ken  et  al.  2000]  satellites. 

The  Lull  Physics-Based  Kalman  111  ter  model  is  based  on  an  ensemble  Kalman  filter  approach 
[Eve  risen,  2003]  and  rigorously  evolves  the  ionosphere  and  plasmasphere  electron  density  field 
and  its  associated  errors  using  a  physics-based  Ionosphere- Plasmasphere  model  (1PM)  [Schunk  et 
al.,  2004,  2005:  Scherliess  et  al,  2004].  The  IPM  is  based  on  a  numerical  solution  of  the  ion  and 
electron  continuity  and  momentum  equations  and  covers  the  low  and  mid-latitudes  from  00  io 
30,000  km  altitude.  In  its  current  version,  the  model  excludes  geomagnetic  latitudes  poleward  of 
&  ±60°  geomagnetic  latitude  due  to  the  vastly  different  physical  processes  that  govern  the  high- 
latitude  regions,  e,g,  convection  electric  fields,  particle  precipitation,  etc.  The  Full  Physics-Based 
data  assimilation  model  provides  specifications  on  a  spatial  grid  that  can  be  global,  regional,  or 
local  and  its  output  includes  the  3-dimensional  electron  and  ion  ("NO'.  G-T,  N/,  Q\  H+,  He  ) 
density  distributions  from  90  km  to  near-geosynchronous  altitude  (30,000  km).  In  addition,  the 
model  provides  the  global  distribution  of  the  ionospheric  drivers  (electric  field,  neutral  wind,  and 
composition)  that  make  the  modeled  state  consistent  with  the  ionospheric  observations.  It  is 
important  to  note  that  the  estimation  of  the  ionospheric  drivers  is  an  integral  part  of  our  ensemble 


147  Kalman  filter  an d  is  achieved  by  using  the  internal  physics- based  model  sensitivities  to  the 

148  various  driving  forces.  In  this  procedure,  die  ionospheric  data  are  used  to  adjust  the  plasma 

149  densities  and  its  drivers  so  that  a  consistency  between  the  observations  (within  their  errors)  and 

150  the  physical  model  is  achieved.  As  a  result  the  assimilation  procedure  produces  the  optimal 

151  model-data  combination  of  the  ionosphere-plasmasphcre  system  together  with  the  set  of  drivers 

152  (electric  fields  and  neutral  winds  and  composition)  consistent  with  the  ionospheric  observations 
158  [Scherliess  et  a!.,  2009,  201 1  ], 

154  4.1  Kalman  Filter  Simulations 

155  The  Full  Physics-Based  data  assimilation  model  was  designed  to  specify  ionospheric  weather, 

156  but  the  model  can  also  be  used  to  study  the  sensitivity  of  the  specification  accuracy  on  different 

157  arrangements  of  observation  platforms  and  observ ation  geometries  and  can  provide  important 

158  information  for  the  planning  of  future  missions  [e.g.  Atlas,  1997;  Atlas  et  aLt  1985].  For  the 

159  current  study  this  latter  mode  has  been  used  and  simulation  experiments  have  been  performed  In 

160  this  mode  the  model  uses  an  Observing  System  Simulation  Experiment  (OSSE)  using  two 

161  different  sy  nthetic  (model-generated)  data  types:  slant  TEC  from  ground-based  GPS  stations  and 

162  in- situ  plasma  density  measurements  obtained  from  electrostatic  analyzers  (ESA)  onboard  of  a 

163  constellation  of  small  satellites.  Figure  1  shows  a  snapshot  of  the  geographic  distribution  of 

164  ground-based  GPS-TEC  observations  shown  at  their  300  km  pierce  points  used  in  this  study.  In 

165  the  OSSE,  the  simulated  weather  (true)  time- dependent  ion  and  electron  density  distributions  are 

166  generated  by  using  again  the  l PM  model.  For  this  study  we  have  used  two  geomagnetically  quiet 

167  days,  20 if)  day  73  and  74  (March  I4lh/15[h)  where  Kp-1  throughout  the  two  days.  These  two 

168  days  were  chosen  to  assist  in  another  study  comparing  actual  MESA  data  from  the  MISSE7 


169  experiment  on  the  ISS  with  the  GAIM  predictions;  that  work  is  outside  the  scope  of  this  paper. 

170  However,  for  the  purposes  of  this  paper  discussing  the  application  of  a  proposed  methodology  , 

171  these  days  may  he  considered  arbitrary';  we  are  not  attempting  to  arrive  at  an  absolute  "truth" 

172  measurement  for  these  days  in  this  paper*  but  rather  apply  a  general  methodology  applicable 

173  regardless  of  the  days  selected  (within  the  applicable  range  of  geomagnetic  conditions  of  the 

174  assimilative  model  used),  the  satellite  constellation  orbital  parameters,  the  specific 

175  ch  arac  tcri  st  ics/pre  c  i  s  i  on  /accuracy  of  th  e  sensor/i  n  strum  en  t  s  u  i  te  used ,  olh  e  r  d  ala  sou  i  ces  i  n  ges  Led 

176  into  GAIM,  etc. 

177  For  the  simulations  we  varied  the  equatorial  vertical  drift  and  horizontal  neutral  winds  by 

178  superposing  on  the  climatology'  values  a  random  component.  At  low  and  middle  latitudes,  tire 

179  important  inputs  into  the  IPM  arc  the  neutral  wind  field  and  the  equatorial  electric  field.  The 

ISO  empirical  horizontal  wind  model  (HWM)  was  used  to  generate  the  neutral  wind  field  and  the 

181  empirical  Scherliess-Fejer  equatorial  vertical  drift  model  was  used  for  the  equatorial  electric 

182  fields.  When  these  empirical  inputs  are  used,  the  IPM  yields  ionospheric  climatology  that  is 

183  consistent  with  the  empirical  inputs.  The  goal  of  the  present  study  is  however  not  to  reproduce 

184  climatology,  hut  to  reproduce  real-time  weather  features.  Consequently,  a  disturbed  ionosphere 

185  ill  at  represents  weather  is  needed  for  the  OSSE.  The  disturbed  ionosphere  was  obtained  by 

186  adding  to  the  climatological  neutral  wind  and  electric  field  values  a  random  component  at  each 

187  1 5-minute  time  step*  For  the  electric  fields  (vertical  drills)  a  random  value  between  ±10  m/s  was 

ISS  chosen  and  for  the  neutral  wind  a  random  value  of  ±50  m/s  was  chosen.  This  1PM  run  was 

189  assumed  to  represent  weather  and  was  taken  to  be  the  truth.  Clearly,  there  are  a  couple  of 

190  important  caveats  associated  with  these  model  runs*  For  example  the  truth  run  only  varied  the 

191  neutral  wind  and  low  latitude  electric  fields  and  did  not.  for  example,  considered  changes  in  the 


132  temperatures  and  other  ionospheric/thermospheric  parameters.  Note  that  neither  the 

193  climatological  values  nor  the  random  components  arc  known  to  the  ensemble  Kalman  filler  part 

194  The  synthetic  data  were  then  generated  by  probing  the  3-D,  time-dependent  electron  density 

195  distribution  for  die  weather  (true)  simulation  exactly  the  same  way  the  real  instruments  probe  the 

196  real  ionosphere.  For  the  GPS  receivers,  slant  TEC  values  were  generated  only  for  elevation 

197  angles  greater  than  15°.  For  the  in-situ  electron  densities  synthetic  observations  were  generated 

198  in  IG-sec  increments.  When  the  synthetic  data  were  generated,  noise  was  added  to  each 

199  “measurement”  in  order  to  mimic  a  real  observation.  A  5  TEC  unit  (TECU)  ievcl  of  noise  was 

200  added  to  all  simulated  TEC  measurements  and  a  10%  uncertainty  to  the  simulated  in -situ 

201  measurements.  It  should  be  noted  that  the  10  second  resolution  of  the  observations  is  not 

202  intended  to  capture  variations  that  occur  on  this  small  lime  step,  but  instead  to  capture  spatial 

203  variations  of  the  order  of  about  100-150  km  around  the  latitudinal  resolution  of  the  model.  10 

204  seconds  is  also  the  nominal  cadence  of  data-taking  in  the  1MESA  instrument  (although  it  is 

205  capable  of  running  as  fast  as  2Hz),  The  satellites  traverse  a  distance  of  about  70  km  in  10 

206  seconds,  which  provides  about  two  observations  per  latitude  grid  cell.  The  model  time  step  of  15 

207  minutes  was  chosen  to  capture  typical  variations  in  the  ionospheric  F-region  where  the 

208  characteristic  timescales  arc  of  the  order  of  tens  of  minutes, 

209  The  ensemble  Kalman  filler  assimilation  procedure  was  implemented  as  follows.  At  0000  UT  on 

210  day  2010/073  the  plasma  distribution  obtained  from  Lhe  'IrutE'  run  (the  l PM  run  with  the 

211  modified  climatological  neutral  wind  and  equatorial  electric  field  input)  was  taken  to  be  the 

212  initial  distribution  at  the  start  of  the  assimilation.  Every  1 5  min,  the  evolving  weather  simulation 

213  was  probed  to  obtain  the  two  synthetic  data  types  (with  noise)  as  described  above.  At  these  time 


214  marks  the  ensemble  of  ionosphere/plasmasphere  model  runs  was  also  integrated  forward  in  lime, 

215  and  the  model  error  covariance  matrix  was  determined  [Setter  Hess  et  aL  2007 j.  Using  the  new 

216  data  and  the  new  error  matrix,  the  ensemble  Kalman  filter  reconstructed  an  updated  estimate  of 

217  the  plasma  distribution  and  its  drivers.  The  new  drift  and  wind  velocities  were  fed  back  into  the 

218  I  PM  and  the  assimilation  was  repeated  at  the  next  15  min  time  mark.  As  time  advanced,  the 

219  ensemble  Kalman  filler  produced  an  estimation  of  the  3-D,  lime- dependent,  plasma  distribution 

220  from  low-  to  mid- latitudes, 

221  To  qualitatively  and  quantitatively  assess  the  impact  of  the  MESA  observations  on  the  plasma 

222  specifications  four  ensemble  Kalman  filter  simulations  were  performed.  Initially,  the  GA1M-FP 

223  model  assimilated  only  the  simulated  global  TEC  data  to  obtain  a  specification  of  the  plasma 

224  density  for  the  two  days.  This  model  simulation  is  referred  to  as  the  "ionospheric  specification'' 

225  and  may  be  compared  with  the  "truth"  measurement  to  determine  the  accuracy  of  the  data 

226  assimilation  model, 

227  Note,  that  neither  the  climatological  neutral  wind  or  the  equatorial  electric  fields  nor  the  imposed 

228  random  variations  added  to  them  are  known  to  the  filter  in  advance.  The  reason  for  this  choice  is 

229  that  on  any  given  day  the  empirical  climatology  values  can  be  far  off  from  the  actual  values  for 

230  this  day.  The  filler  instead  starts  from  a  zero  value  for  both  the  drifts  and  neutral  winds  and 

231  estimates  the  required  values  for  the  given  day.  With  this,  the  filter  would  not  have  had  good 

232  convergence  if  no  data  would  have  been  assimilated, 

233  Next,  the  synthetic  MESA  "observation",  including  an  observational  uncertainty,  were 

234  assimilated  together  with  die  synthetic  global  TEC  data  to  simulate  data-taking  from  satellite 

235  constellations  that  did  not,  in  reality,  exist  at  that  particular  lime.  The  simulated  observations 
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were  taken  along  satellite  tracks  for  three  different  orbital  scenarios  at  10  second  simulated 
cadence.  The  orbital  scenarios  were  chosen  to  give  both  low-cost  (single  launch)  rapid  re- 
coverage  of  a  localized  area,  and  higher-cost  (multiple  launches)  global  coverage.  We  also 
wished  to  investigate  effect  of  varying  the  altitude  of  the  satellites  on  the  sensitivity  of  the 
specification.  Accordingly,  the  three  orbital  scenarios  chosen  were  as  follows: 

Scenario  A:  Ten  satellites  in  a  circular  500  km  altitude  polar  orbit  (90°  inclination).  Small 
(deliberate)  variations  in  the  satellite  surface  treatment  will  lead  to  small  variations  in 
satellite  drag,  causing  the  satellites  to  distribute  themselves  along  the  orbital  path  until  they 
are  spread  evenly  along  the  orbit.  This  represents  the  "string  of  pearls''  configuration  for  a 
single  launch/deployment. 

Scenario  D:  As  scenario  A,  but  at  a  350  km  altitude. 

Scenario  C:  A  25/5/1  Walker  constellation  at  510  km  altitude  and  60°  inclination.  The 
Walker  constellation  notation  of  t/p/f  [Wertz.  2006]  designates  /  satellites  arranged  over  p 
evenly  spaced  orbital  planes  (circular  orbits)  with  /  relative  spacing  between  satellites  in 
adjacent  planes.  Thus,  a  25/5/1  constellation  has  25  satellites,  five  satellites  per  orbital  plane. 
We  have  picked  a  nominal  relative  spacing  in  this  instance.  This  would  generally  require  five 
launch  vehicles,  each  deploying  live  satellites,  with  each  group  deploying  into  the  string  of 
pearls  configuration  alter  some  weeks.  Figure  2  shows  (lower  panel)  Ne  at  510  km,  and 
< upper  panel)  the  ground  tracks  of  a  25/5/1  Walker  constellation  with  the  color  scale  along 
each  ground  track  demonstrating  the  sampling  of  Ne. 


256  For  each  of  the  three  orbital  scenarios  the  GAIM-FP  model  was  used  together  with  the  global 

257  TEC  inputs  and  the  MESA  "observation”  to  obtain  another  set  of  specifications  of  the  same  day, 

258  These  model  simulations  arc  referred  to  as  the  'improved  ionospheric  specification”*  and  may  be 

259  compared  to  the  original  "ionospheric  specification”  (without  any  MESA  data  inputs)  and  the 

260  original  "truth”  model  run, 

281  5.  Results 

262  For  Scenarios  A  and  B,  we  have  examined  a  vertical  slice  of  the  ionosphere  from  100  km  to  600 

263  km  altitude  along  the  16E25a  E  line  of  longitude  of  the  orbital  plane,  stretching  from  60°  N  to 

264  60°  S.  (Plots  are  plotted  to  the  poles,  but  the  GAIM  model  used  only  extends  to  ±60°  magnetic 

265  latitude).  Figure  3  shows  the  deviation  from  "truth”  in  units  of  AN e  (cm-3)  lor  three  simulation 

266  runs:  "ionospheric  specification”  using  only  GPS-TEC  inputs  to  the  assimilative  forecast  model 

267  (left  panel),  "improved  ionospheric  specification”  using  G3*S-TEC  inputs  plus  inputs  from 

268  satellites  in  scenario  A  (center  panel)  (satellites  at  500  km),  and  "improved  ionospheric 

269  specification”  using  GPS-TEC  inputs  plus  inputs  from  satellites  in  scenario  B  (right  panel) 

270  (satellites  at  350  km). 

271  A  visual  inspection  of  the  data  shows  that  utilizing  inputs  from  a  modest  constellation  of  ten 

272  satellites  at  either  of  two  altitudes  shows  a  distinct  improvement  to  the  "improved  ionospheric 

273  specification”  (ANe  converges  towards  zero).  What  is  perhaps  remarkable  is  that  the 

274  improvement  is  distinct  at  most  latitudes  and  can  be  seen  at  all  altitudes  and  not  just  at  the  orbital 

275  altitude.  The  apparent  propagation  of  information  to  other  altitude  regimes  is  very  likely  a 

276  manifestation  of  the  strong  correlation  of  electron  density  variations  along  geomagnetic  field 

277  lines.  These  correlations  are  pan  of  the  ensemble  Kalman  filter  and  automatically  calculated 


278  using  the  ensemble  of  physics-based  model  runs.  This  indicates  that  the  useful  life  of  such  a 

279  constellation,  as  orbital  drag  decays  the  orbit  from  the  initial  insertion,  will  be  extended  through 

280  the  life  of  the  mission  (months  to  years  for  higher  initial  orbital  insertions),  rather  than  losing 

281  their  use  after  initial  orbital  decay, 

282  Scenario  C  allows  inspection  of  global  (latitude-longitude)  coverage.  We  have  elected  to  use 

283  hmF2  as  a  proxy  for  our  knowledge  of  the  ionosphere  (remembering  that  although  the  satellites 

284  in  Scenario  C  are  at  500  km,  the  earlier  results  indicate  that  information  is  apparently 

285  propagating  vertically  {most  likely  associated  with  the  correlation  of  electron  density  along 

286  magnetic  field  lines  in  the  ensemble  Kalman  filter)  allowing  us  to  inspect  the  ionosphere  and  the 

287  improvement  to  the  plasma  specifications  at  any  altitude).  We  inspect  hmF2  at  an  arbitrary  time 

288  of  1600  GMT  on  day  2010/073,  Figure  4  shows  limb 2  from  the  "truth”  model  (plotted  to  the 

289  latitude  limits  of  the  modell  at  the  selected  time.  Figure  5  shows  hmF2  obtained  from  the 

290  "ionospheric  specification”  model  using  only  GPS-TEC  inputs  to  the  model.  A  visual  inspection 

291  shows  that  there  are  deviations  from  "truth  '.  In  particular,  the  height  enhancement  over  the 

292  Japanese  sector  is  not  found:  there  is  an  erroneous  equatorial  plume  forecast  over  the  Indonesian 

293  sector;  and  the  pronounced  equatorial  anomaly  over  the  South  American  sector  is  not  seen  in  die 

294  data  assimilation  results  for  this  limbed  model  run. 

295  Figure  6  shows  hml;2  obtained  from  the  "improved  ionospheric  specification”,  utilizing  GPS- 

296  TEC  inputs  plus  simulated  data  from  ihe  MESA  instruments  in  the  Walker  constellation  of 

297  scenario  C.  Inspection  shows  that  the  three  features  mentioned  above,  reproduced  poorly  by  the 

298  "ionospheric  specification”,  are  present  in  the  "improved  ionospheric  specification”. 


299  The  previous  examples  are  qualitative.  There  are  many  quantitative  metrics  that  can  be  used  Lo 

300  quantify  the  improvement,  and  we  have  arbitrarily  selected  two  metrics,  the  RMS  deviation 

301  (global  sum)  and  the  Skill  Score  (global  sum).  The  RMS  deviation,  summed  over  all  latitude- 

302  longitude  points  at  each  hmF2  altitude,  is  given  by  the  sum  of  the  squares  of  the  deviations  from 

303  truth  divided  by  the  number  of  observations  and  is  measured  in  km  (equation  I  ).  Improved 

304  modeling  will  reduce  the  RMS  deviation  towards  zero. 

305  The  Skill  Score  is  a  measure  of  improvement  of  one  model  over  another  and  is  unitless,  ranging 

306  from  -infinity  to  +1 .  If  the  second  model  is  perfect  the  Skill  Score  tends  to  +1,  and  if  the  second 

307  model  predicts  no  better  than  random  chance  the  Skill  Score  tends  towards  0  (equation  2). 


308  RMS=  jz<Tru,h  ~.obs)l  (]) 

N 


309  Skill  Score  = 


(GPS+1MESA})* 
Z(Truth-GPS  GiUy)2 


(2) 


310  Figure  7  shows  the  RMS  deviation  from  truth  global  sum  over  a  24-hour  period  on  2010  day 

3n  073,  With  the  "ionospheric  specification”  (GPS-TEC  only),  the  RMS  deviation  from  truth  varies 

312  between  25  km  and  1 00  km  over  the  course  of  the  24  hours.  It  is  interesting  to  note  the  variation 

313  over  time,  and  we  suggest  that  this  Ls  due  a  function  of  daytime  and  nighttime  regions  being 

314  densely  or  less  densely  populated  with  GPS-TEC  ground  stations,  as  the  sun  moves  from  the 

315  Pacific  sector  (less  densely  populated  with  GPS-TEC  ground  stations)  to  the  American  and 

316  European  sectors  (more  densely  populated).  With  the  "improved  ionospheric  specification” 

317  (GPS-TEC  plus  simulated  MESA  data  from  scenario  C),  which  of  course  are  agnostic  to  day- 


3 is  night  variations,  the  RMS  deviation  global  sum  improves  to  around  10  km  over  the  course  of  the 

319  day, 

320  The  Skill  Score  comparison  of  the  two  model  runs  is  shown  in  Figure  Again  we  see  a 

321  pronounced  diurnal  response,  with  the  ‘improved  ionospheric  specification'1  performing 

322  extremely  well  in  the  0(H)0  LIT  to  0600  LIT,  and  1800  LIT  to  2359  UT  timeframes,  and 

323  marginally  less  well  between  those  timeframes.  However  with  the  Skill  Score  minimum  being 

324  -0,75,  we  can  conclude  that  overall  there  is  a  marked  improvement  to  our  forecast  model  with  25 

325  MESAs  in  orbital  scenario  C 

326  We  recognize  that  the  results  obtained  from  these  metrics  can  depend  on  other  parameters  such 

327  as  geomagnetic  activity,  hut  such  an  investigation  is  outside  the  bounds  of  this  paper. 

328  6.  Conclusions 

329  We  have  proposed  using  a  simple  sensor  that  measures  ion  and  electron  energy  spectra,  from 

330  which  plasma  density  and  temperature  can  he  derived,  in  a  low-cost  mission  of  small 

331  satellites/sensors.  A  full  phy'sics  ionospheric  model  has  been  utilized  to  derive  "truth"  data.  The 

332  GAIM-FP  data  assimilation  model  has  then  been  run  both  without  and  with  simulated  sensor 

333  data  inputs.  Our  model  simulations  have  shown  that  adding  a  constellation  of  small 

334  said  liles/sensors  in  addition  to  global  TEC  inputs  converges  the  GAIM-FP  model  closer  to 

335  "truth''1  in  the  situations  we  describe.  For  a  real -life  mission  for  which  a  launch  has  not  been 

336  imposed  a  desired  improvement  metric  may  be  selected  and  thus  orbital  parameters  can  be  fine- 

337  tuned  to  optimize  the  model  improvement  for  the  metric  of  interest.  What  is  particularly 

338  interesting  is  dial  the  model  is  improved  over  a  range  of  altitudes,  not  just  at  and  around  the 


339  satellite/ sensor  altitude,  emphasizing  the  coupled  nature  of  both  the  model  and  reality.  Put 

340  another  way,  know  ledge  at  one  location  leads  to  improved  knowledge  at  other  locations. 

341  However,  what  has  become  apparent  is  the  challenge  to  develop  a  generally  agreed  metric  or  set 

342  of  metrics  to  measure  the  scientific  and  operational  benefit  to  assimilative  models  from  the  use 

343  of  multiple  small  satellite/ sensor  inputs. 
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427  Figure  I:  Geographical  distribution  of  ground-based  GPS-TEC  observations  chosen  for  this 

428  study.  The  figure  shows  the  300  km  pierce  points  of  observations  at  the  given  time  (20 10/073 

429  1200  UT),  and  the  color  scale  depicts  vertical  TEC  with  blue  indicating  low  TEC  and  red 

430  i  n  d  i  cat i  ng  h  i  gh  TEC . 

431 

432  Figure  2:  (upper  panel)  25/5/1  Walker  constellation  ground  tracks  showing  sampling  ofNe  from 

433  "truth"  model  run,  and  (lower  panel)  snapshot  of  Ne  obtained  from  an  "improved  ionospheric 

434  specification"  forecast  model  (GPS-TEC  and  sampled  MESA  data). 

435 

436  Figure  3:  Model  deviations  from  'irutfT  run  for  scenarios  A  and  B,  over  die  altitude-latitude 

437  slice  at  longitude  16L25*  E.  Changes  in  Ne  are  denoted  by  variations  in  the  color  scale  from 

438  green  (no  deviation). 

439 

440  Figure  4:  Snapshot  of  hml2  for  the  "Truth"  model. 

441 

442  Figure  5;  Snapshot  of  hmF2  for  the  "ionospheric  specification"  model  assimilating  only  GPS- 

443  TEC  into  the  GAIM-FP  model. 


444 


445  Figure  6:  Snapshot  of  hmF2  for  the  "improved  ionospheric  specification"  model  assimilating 

446  GPS-TEC  and  in-situ  MESA  data  into  GA1M-FP. 

447 

44S  Figure  7:  RMS  deviation  of  hmF2  prediction  (global  sum).  The  upper  (dotted)  line  shows 

449  predictions  from  the  "ionospheric  specification"  (GPS-TEC  only);  the  lower  (solid)  plot  shows 

450  the  predictions  for  "improved  ionospheric  specification"  (GPS-TEC  and  MESA  inputs). 

451 


452  Figure  8:  Skill  Score  of  hmF2  prediction,  comparing  "improved  ionospheric  specification"  to 

453  "ionospheric  specification". 
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